Immersive virtual reality detection and alerting technology

ABSTRACT

Methods, systems, and apparatus, including computer programs encoded on a storage device, for alerting a user immersed in a virtual reality environment to the occurrence of an event. In one aspect, a system includes sensors and a monitoring unit. The monitoring unit includes a network interface, a processor, and a storage device that includes instructions to cause the processor to perform operations. The operations include detecting the occurrence of an event at a property, determining that a user who is located at the property is immersed in a virtual reality environment at a time of occurrence of the event at the property, based on determining that the user who is located at the property is immersed in the virtual reality environment at the time of the occurrence of the event, generating an alert notification that is configured to initiate the user&#39;s emergence from the virtual reality environment.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/851,704, filed Dec. 21, 2017, now allowed, which claims the benefitof U.S. Provisional Patent Application No. 62/437,517 filed Dec. 21,2016. Both of these prior applications are incorporated by reference intheir entirety.

BACKGROUND

The availability and use of virtual reality devices is becoming moreprevalent. Such devices are capable of generating an artificialenvironment that can distract a user of the virtual reality device ofevents in the real-world.

SUMMARY

The subject matter of the present disclosure is related to techniquesfor using a monitoring system to transmit event notifications toequipment that is used to create a virtual reality environment. Forexample, the monitoring system is configured to generate and usespecific types of notifications designed get a user of virtual realityequipment to emerge from a virtual reality environment. Equipment thatis used to create a virtual reality environment (or virtual realityequipment) can include one or more computers that are used to generate avirtual reality environment including, for example, one or more servers,one or more desktop computers, one or more laptop computers, one or morecameras, a smartphone, a virtual reality headset, or a combinationthereof.

In some implementations, upon detection of an event (e.g., a securityevent, an emergency event, or other type of alarm event), the monitoringsystem may be configured to automatically transmit a notification toequipment that is used to create a virtual reality environment.Alternatively, in other implementations, upon detection of an event, themonitoring system may be configured to determine whether the user isimmersed in a virtual reality environment. Then, if the user isdetermined to be immersed in a virtual reality environment, the systemmay transmit a notification to equipment that is used to create thevirtual reality experience in an effort to get the user to emerge fromthe virtual reality environment. Types of notifications includenotifications that (i) trigger playback of audio sounds in the VRheadset, (ii) display video alerts on the display of the VR headset,(iii) cause the headset to vibrate, (iv) power down the VR headset, (v)instruct a drone to fly over and nudge the user's leg, (vi) instruct adrone to walk (or roll) over and nudge the user's leg, or the like.

According to one innovative aspect of the present disclosure, amonitoring system is disclosed for alerting a user immersed in a VRenvironment to the occurrence of an event at a property. The monitoringsystem may include a plurality of sensors located at a property and amonitoring unit that is configured to obtain sensor data generated byone or more of the plurality of sensors located at the property. Themonitoring system may include a network interface, one or moreprocessors, and one or more storage devices that include instructionsthat are operable, when executed by the one or more processors, to causethe one or more processors to perform operations. The operations mayinclude detecting the occurrence of an event at the property,determining, based on the obtained sensor data, that a user who islocated at the property is immersed in a virtual reality environment ata time of the occurrence of the event at the property, and based ondetermining that the user who is located at the property is immersed inthe virtual reality environment at the time of the occurrence of theevent at the property, generating an alert notification that isconfigured to initiate the user's emergence from the virtual realityenvironment

Other aspects include corresponding methods, apparatus, and computerprograms to perform actions of methods defined by instructions encodedon computer storage devices.

These and other versions may optionally include one or more of thefollowing features. For example, in some implementations, the operationsmay include transmitting the generated alert notification to the virtualreality device.

In some implementations, the operations may further include determiningan importance score for the detected event, determining that theimportance score for the detected event satisfies a predeterminedthreshold, based on determining (i) that the user who is located at theproperty is immersed in the virtual reality environment and (ii) thatthe importance score satisfies the predetermined threshold, generatingan alert notification that is configured to initiate the user'semergence from the virtual reality environment.

In some implementations, the operations may include transmitting thegenerated alert notification to the virtual reality device.

In some implementations, the system may further include the virtualreality device that includes a virtual reality headset. In someimplementations, the virtual reality headset may include (i) headwearthat includes a receiving unit configured to receive a smartphone and(ii) a smartphone.

In some implementations, obtaining sensor data generated by one or moreof the plurality of sensors may include detecting sensor data that isgenerated and broadcast by one or more motion sensors, one or morecontact sensors, one or more glass break sensors, one or more biometricsensors, one or more sensors worn by the user, one or more temperaturesensors, one or more smoke sensors, or one or more carbon monoxidesensors, and determining, based on the obtained sensor data, that a useris immersed in a virtual reality environment may include determining,based on the detected sensor data that is generated and broadcast by theone or more motion sensors, the one or more contact sensors, the one ormore glass break sensors, the one or more biometric sensors, the one ormore sensors worn by the user, the one or more temperature sensors, theone or more smoke sensors, or the one or more carbon monoxide sensors,that a user is immersed in a virtual reality environment.

In some implementations, determining, based on the obtained sensor data,that a user who is located at the property is immersed in a virtualreality environment at a time of the occurrence of the event at theproperty may include determining, based on the obtained sensor data,that (i) the user's location is stationary and (ii) one or more arms ofthe user and the head of the user is moving.

In some implementations, one or more of the plurality of sensors mayinclude one or more sensors that are located on the user's body. In suchimplementations, determining, based on the obtained sensor data, that auser who is located at the property is immersed in a virtual realityenvironment at a time of the occurrence of the event at the property mayinclude determining, based on (i) obtained sensor data that is generatedand broadcast by the one or more that are located on the user's body and(ii) the occurrence of the event, that the user is immersed in thevirtual reality environment.

In some implementations, the one or more sensors worn by the user mayinclude a biometric sensor or a motion sensor.

In some implementations, determining, based on the obtained sensor data,that a user who is located at the property is immersed in a virtualreality environment at a time of the occurrence of the event at theproperty may include determining, based on the sensor data that the userresides in a particular portion of a property, determining, based on thesensor data, that the event is occurring within a predetermined distanceof the particular portion of the property, and determining that the useris immersed in the virtual reality environment based on thedetermination that (i) the user is resides in the particular portion ofthe property and (ii) the event is occurring within the predetermineddistance of the particular portion of the property.

In some implementations, generating an alert notification that isconfigured to initiate the user's emergence from the virtual realityenvironment may include generating one or more instructions that, whenexecuted by the virtual reality device, instruct the virtual realitydevice to output an audio message through a speaker of the virtualreality device that notifies the user of the event.

In some implementations, generating an alert notification that isconfigured to initiate the user's emergence from the virtual realityenvironment may include generating one or more instructions that, whenexecuted by the virtual reality device, instruct the virtual realitydevice to display a visual message on a display of the virtual realitydevice that notifies the user of the event.

In some implementations, generating an alert notification that isconfigured to initiate the user's emergence from the virtual realityenvironment may include generating one or more instructions that, whenexecuted by the virtual reality device, instruct the virtual realitydevice to vibrate.

In some implementations, generating an alert notification that isconfigured to initiate the user's emergence from the virtual realityenvironment may include generating one or more instructions that, whenexecuted by the virtual reality device, instruct the virtual realitydevice to power down.

In some implementations, determining that the transmitted alertnotification failed to initiate the user's emergence from the virtualreality environment and in response to determining that the transmittedalert notification failed to initiate the user's emergence from thevirtual reality environment, generating one or more instructions that,when executed by the virtual reality device, instruct the virtualreality headset to power down.

In some implementations, detecting the occurrence of an event mayinclude determining, based on an entry of a calendar associated with theuser, that the user has an upcoming appointment. In suchimplementations, generating an alert notification that is configured toinitiate the user's emergence from the virtual reality environment mayinclude in response to determining, based on the entry in the calendarassociated with the user, that the user has an upcoming appointment,generating an alert notification to initiate the user's emergence fromthe virtual reality environment.

In some implementations, detecting the occurrence of an event mayinclude determining, based on sensor data generated and broadcast by asmoke detector or temperature sensor, that a fire is occurring at aproperty. In such implementations, generating an alert notification thatis configured to initiate the user's emergence from the virtual realityenvironment may include in response to determining, based on the sensordata generated and broadcast by the smoke detector or the temperaturesensor, that a fire is occurring at the property, generating an alertnotification to initiate the user's emergence from the virtual realityenvironment.

In some implementations, detecting the occurrence of an event mayinclude determining, based on sensor data generated and broadcast by acontact sensor, that a door or window has opened at a property. In suchimplementations, generating an alert notification that is configured toinitiate the user's emergence from the virtual reality environment mayinclude in response to determining, based on sensor data generated andbroadcast by a contact sensor, that a door or window has opened at theproperty, generating an alert notification to initiate the user'semergence from the virtual reality environment.

In some implementations, detecting the occurrence of an event mayinclude determining, based on data generated and broadcast by aconnected doorbell, that a person is at the door or window. In suchimplementations, generating an alert notification that is configured toinitiate the user's emergence from the virtual reality environment mayinclude in response to determining, based on the data generated andbroadcast by the connected doorbell, generating an alert notification toinitiate the user's emergence from the virtual reality environment.

In some implementations, the monitoring unit is located in a remotelocation from property.

In some implementations, determining that the transmitted alertnotification was unsuccessful in causing the user to emerge from thevirtual reality environment and in response to determining that thetransmitted alert notification was unsuccessful in causing the user toemerge from the virtual reality environment, generating a subsequentnotification that is configured to instruct a drone to (i) navigate to alocation that is associated with the user and (ii) make contact with theuser.

According to another innovative aspect, a method disclosed for alertinga user immersed in a VR environment to the occurrence of an event at aproperty. In one aspect, the method may include actions of detecting theoccurrence of an event at the property, determining, based on theobtained sensor data, that a user who is located at the property isimmersed in a virtual reality environment, based on determining that theuser who is located at the property is immersed in the virtual realityenvironment, generating a subsequent notification that is configured toinstruct a drone to (i) navigate to a location that is associated withthe user and (ii) make contact with the user in order to initiate theuser's emergence from the virtual reality environment, and transmittingthe generated alert notification to the drone.

The monitoring system described by the present disclosure providesmultiple advantages over existing systems. For example, the monitoringsystem of the present disclosure can obtain and analyze sensor datagenerated by one or more sensors at the property to determine whether anoccupant of the property is immersed in a virtual reality environment.Such a system does not require notifications from a virtual realitydevice indicating that a user is immersed in a virtual realityenvironment. Instead, the monitoring system can infer that a user isimmersed in a virtual reality environment based on the sensor data. Thiscan help to conserve network bandwidth because polling messages are notbeing transmitted by the virtual reality device to a monitoring unit ofthe monitoring system to inform the monitoring unit of the status of thevirtual reality device. In addition, the monitoring system can detect,based on the sensor data whether an occupant is immersed in a virtualreality environment without the occupant having to inform the monitoringunit that the occupant is going to immerse himself or herself in avirtual reality environment.

The monitoring system of the present disclosure provides otheradvantages that are apparent from the detailed description below andfrom the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a contextual diagram of a system for alerting a user immersedin a virtual reality environment of the occurrence of an event that wasdetected at a property.

FIG. 2 is another contextual diagram of a system for alerting a userimmersed in a virtual reality environment of the occurrence of an eventthat was detected at a property.

FIG. 3 is a flowchart of a process for alerting a user immersed in avirtual reality environment of the occurrence of an event that wasdetected at a property.

FIG. 4 is a block diagram of an example of a security monitoring system400 that may utilize virtual reality components.

DETAILED DESCRIPTION

FIG. 1 is a contextual diagram of a system for alerting a user immersedin a virtual reality environment of the occurrence of an event that wasdetected at a property. The system 100 may include one or moremonitoring system components such as a monitoring system control unit110, one or more sensors 120 a, 120 b, 120 c, 120 d, 120 e, 120 f, 120g, 120 h, 120 i, one or more cameras 130 a, 130 b, 130 c, 130 d, 130 e,130 f, one or more wireless charging stations 140 a, 140 b, a localwireless network 150, a virtual reality headset 160, a drone 170, aremote network 190, one or more communication links 192, a monitoringapplication server 195, or any combination thereof.

A monitoring system 100 may be employed in accordance with aspects ofthe present disclosure that does not include all of the aforementionedmonitoring system components. In some implementations, for example, onlya subset of the aforementioned components may be used to alert a user ofvirtual reality equipment in an attempt to initiate the user's emergencefrom a virtual reality experience in response to a detected event. As anexample, there may be implementations that use less cameras thandepicted by FIG. 1, more cameras than depicted by FIG. 1, or even nocameras. Similarly, for example, there may be implementations of themonitoring system 100 that use a drone, multiple drones, flying drones,land-based drones, or even no drones. Similarly, for example, there maybe implementations that use less wireless charging stations, morewireless charging stations, no wireless charging stations, or the like.Yet other alternative implementations monitoring system 100 also fallwithin the scope of the present disclosure such as a system that doesnot use a monitoring application server 195. For these reasons, thesystem 100 should not be viewed as limiting the present disclosure toany particular set of necessary components.

The monitoring system control unit 110 obtains sensor data that isgenerated by one or more sensors 120 a, 120 b, 120 c, 120 d, 120 e, 120f, 120 g, 120 h, 120 i. The monitoring system control unit 110 mayanalyze the obtained sensor data in order to determine whether one ormore potential events are occurring in the property 101, within apredetermined proximity of one or more portions of the property 101, orboth. Alternatively, in other implementations, the monitoring systemcontrol unit 110 may relay the obtained sensor data to the monitoringdata to application server 195 using the networks 150, 190, and one ormore communication links 192. In such implementations, the monitoringapplication server may analyze the obtained sensor data in order toinfer whether one or more potential events are occurring in the property101. The monitoring application server 195 may then transmit acommunication to the monitoring system control unit 110 indicatingwhether an event is detected in the property 101. In general, themonitoring application server 195 may also operate as a remotemonitoring unit that can perform any of the operations described hereinwith respect to the monitoring system control unit 110.

Events that may be detected based on the obtained sensor data mayinclude, for example, alarm events, emergency events, family events,calendar events, or the like. Alarm events may be inferred based on oneor more sensors such as sensors 120 a, 120 b, 120 c, 120 d, 120 e, 120f, 120 g, 120 h, 120 i generating data that is indicative of a potentialintruder breaking-and-entering into property 101, a trespassertrespassing within a predetermined proximity of property 101, or thelike. Emergency events may be inferred based on one or more sensors suchas sensors 120 a, 120 b, 120 c, 120 d, 120 e, 120 f, 120 g, 120 h, 120 igenerating data that is indicative of a potentially hazardous event suchas the presence of smoke, fire, carbon monoxide (or other gas), or thelike. Family events may be inferred based on one or more sensors such assensors 120 a, 120 b, 120 c, 120 d, 120 e, 120 f, 120 g, 120 h, 120 iindicating that a family member such as a husband, wife, sibling, kid orother legitimate occupant of the property 101 has entered the property101. Calendar events may be inferred by the monitoring system controlunit 110 accessing a user's 105 calendar maintained on one or more userdevices, and determining that the user 105 has an upcoming appointment.The detections of yet other types of events may also fall within thescope of the present disclosure.

A user 105 may use a virtual reality headset 160 to immersehimself/herself into a virtual reality environment. The virtual realityheadset 160 may include a display, one or more processing units, one ormore memory units, one or more speakers, one or more microphones, or acombination thereof. In some implementations, the virtual realityheadset 160 may be headwear that is configured to receive a user devicesuch as a smartphone, and the user device can be used to create thevirtual reality environment.

The virtual reality headset 160 may create an immersive virtual realityenvironment by generating a multi-dimensional image (e.g., a threedimensional image) that appears to surround the user 105. The virtualreality environment created by headset 160 may engage all, or a subset,of the user's 105 senses so that the user is tricked into forgettingthat he/she is located at property 101 and instead makes the useractually feel like the user is in perceive that the user 105 is immersedin the virtual reality environment. This can include one or more of (i)creating visual effects to change what the user 105 sees, (ii) creatingaudio effects to change what the user 105 hears, (iii) creating smellsto change what the user 105 smells, (iv) a combination thereof, or thelike. Accordingly, while the user 105 is immersed in a virtual realityenvironment, the user 105 may not be aware of any potential events thatmay be happening in the user's immediate physical surroundings ofproperty 101 because the user's 105 senses are distracted by theimmersive virtual reality environment that the user 105 is experiencing.

The types of events that may be detected can vary in severity based onthe types of potential events that are detected. For instance, somepotential events may be low-risk events that are not fatal. Forinstance, a low-risk event may include a family event (e.g., kidsarriving home from school, wife arriving home from work, or the like).Alternatively, some potential events may be high-risk events that mayhave the potential to be fatal. For example, a high-risk event mayinclude an emergency event (e.g., a carbon monoxide leak, a fire, or thelike), an alarm event (e.g., a breaking and entering, trespassing in theperimeter of the property, or the like). It is also contemplated thatsome potential events may be moderate-risk events that are not fatal butextremely important such as some calendar events (e.g., a meeting with aclient, a meeting with a supervisor, or the like).

Upon the detection of such events, the monitoring system control unit110 may determine whether a user 150 is immersed in a virtual realityenvironment. Determining whether a user 150 is immersed in a virtualreality environment may include, for example, obtaining sensor datagenerated by one or more sensors located throughout the property 101such as sensors 120 d, 120 e, 120 f, 120 g and analyzing the obtainedsensor data. For example, the sensor data may include data that isindicative of a user 105 using of a virtual reality headset 160. Forinstance, the obtained sensor data may indicate that a user 105 ispresent in the property 101, remaining substantially stationary in aparticular portion of property 101, but moving the user's 105 head andarms. Alternatively, or in addition, the obtained sensor data mayinclude data from one or more sensors 120 i worn by the user 105. Forexample, the user 105 may wear a smartwatch, or other wearable device,having one or more sensors that measure, generate, broadcast, or acombination thereof, (i) sensor data associated with the user's healthsuch as a user's heart rate, blood pressure, breathing rate, or thelike, (ii) sensor data associated with the user's body parts such aswhether a user's arm, leg, or other body part is moving, or (iii) thelike. Alternatively, or in addition, the monitoring system control until110 may receive notifications from the virtual reality headset 160 viathe network 150 when the virtual reality headset is being used by theuser 105.

In some implementations, a combination of sensor data from multipledifferent sensors 120 a, 120 b, 120 c, 120 d, 120 e, 120 f, 120 g, 120h, 120 i may be analyzed to detect whether a user 105 is currentlyimmersed in a virtual reality environment. For instance, the monitoringsystem control unit 110 (or monitoring application server 195) mayanalyze (i) sensor data from sensors 120 d, 120 e, 120 f, 120 g in RoomD that is indicative of the user's 105 presence in Room D, degree ofmovement, or both, (ii) sensor data from one or more wearable sensors120 i worn by the user 105 that may provide, e.g., biometric data (e.g.,heart rate) related to the user 105, and (iii) sensor data from a sensor120 a such as a smoke detector in room A and determine whether a user isimmersed in a virtual reality environment. For example, if the sensordata from Room D indicates that the user is in Room D, the sensor datafrom wearable sensor 120 i indicates that the user's heart rate isindicative of a person who is alive and awake, and one or more smokesensors or the like indicate the presence of smoke in property 101, themonitoring system control unit 110 (or monitoring application server195) may conclude that the user 105 is immersed in a virtual realityenvironment because the sensor data indicates that there is a fire andthe user 105 is in an adjacent room alive, awake, and not reacting tothe fire. This may be because, for example, the user 105 cannot see thefire, hear the fire, feel the heat from the fire, or smell the smokefrom the fire when immersed in the virtual reality environment createdby the virtual reality headset 160. Other types of sensor data may beanalyzed in a variety of different ways by the monitoring system controlunit 110 (or monitoring application server 195) in order for themonitoring system control unit 110 (or monitoring application server195) to determine that the user 105 is immersed in a virtual realityenvironment.

Once the monitoring system control unit 110 (or monitoring applicationserver 195) identifies a potential event and determines that the user105 is immersed in an immersive virtual reality environment created byvirtual reality headset 160, the monitoring system control unit 110 mayinitiate actions to notify equipment creating a virtual realityenvironment in an attempt to trigger user 105 emergence from the virtualreality environment. In particular, the monitoring system control unit110 may generate one or more alert notifications 112, 114. The alertnotifications 112, 114 may be transmitted to virtual reality equipmentcreating a virtual reality environment such as the virtual realityheadset 160, a server running virtual reality video that is beingstreamed to the virtual reality headset 160, or the like. The alertnotifications 112, 114 may be configured to instruct (i) the virtualreality equipment to get the user 105 to emerge from the immersivevirtual reality environment created by the virtual reality headset 160,(ii) a virtual reality server streaming video to the virtual realityheadset, or (ii) the like to address the potential event detected by themonitoring system control unit 110 (or monitoring application server195). Addressing the potential event may include, for example, emergingfrom the virtual reality environment to (i) meet the user's 105 kidsafter the kids come home from school, (ii) place a phone call to theuser's 105 supervisor, (iii) use a fire extinguisher to put out a fire,(iv) confront a burglar, (v) flee the property 101, or the like.

Though the system 100 is described above with the monitoring systemcontrol unit 110 initiating actions to notify equipment creating avirtual reality environment only after the monitoring system controlunit 110 determines that the user 105 is immersed in a virtual realityenvironment, the present disclosure need not be so limited. In someimplementations, the monitoring system control unit 110 can initiateactions to notify equipment creating a virtual reality environment assoon as a potential event is detected. By way of example, upon thedetection of more than a threshold amount of carbon monoxide in the air,the monitoring system control unit 110 may immediately transmit anotification to equipment creating a virtual reality environment toinstruct the equipment to trigger a user's 105 emergence from thevirtual reality environment without first determining whether the user105 is immersed in a virtual reality environment.

The monitoring system control unit 110 may generate different types ofmessages that notify the user of the potential event and attempt to getthe user 105 to emerge from the immersive virtual reality environment indifferent ways. For instance, the monitoring system control unit 110 maygenerate an alert notification 112 that is transmitted to the virtualreality headset 160. The alert notification 112 may include one or moreinstructions that instruct the virtual reality headset 160 to alert theuser 105. For instance, the alert notification 112 may instruct thevirtual reality headset 160 to provide a message for display on thegraphical user interface of the virtual reality headset indicating thepresence of an event, data describing the event, or both. Alternatively,or in addition, the alert notification 112 may instruct the virtualreality headset 160 to play an audio message notifying the user 105 ofthe potential event. Alternatively, or in addition, the alertnotification 112 may be transmitted to a computer system (e.g., aserver) that is networked to the virtual reality headset that instructsthe computer system to pass-through a visual message, audio message, orthe like to the virtual reality headset in an effort to notify the userof the potential event. Alternatively, or in addition, the alertnotification 112 may instruct the virtual reality headset 160 to vibratein an effort to inform the user 105 of the occurrence of a potentialevent. Alternatively, or in addition, the alert notification 112 mayinstruct the virtual reality headset 160 to power down. An alertnotification 112 that includes an instruction for the virtual realityheadset 160 to power down may be transmitted to the virtually realityheadset 160 in certain instances where previous alert notifications havefailed to get the user 105 to emerge from the immersive virtual realityenvironment.

Other types of alert notifications may include, for example, an alertnotification 114. The alert notification 114 may be transmitted to adrone 170. The alert notification 114 may include an instruction thatinstructs the drone 170 to navigate to the user's 105 location andperform an action at the user's 105 location. The drone 170 may navigatealong a flight path 172 to the user's 105 location. The user's 105location may be determined based on one or more sensors such as sensors120 d, 120 e, 120 f, 120 g, 120 i.

Once the drone 170 arrives that the user's 105 location, the drone 170may make contact with the user 105 in an effort to initiate the user's105 emergence from a virtual reality environment. Making contact withthe user 105 may include, for example, the drone 170 using an armdeployment mechanism 170 a to deploy robotic arm 170 b that can be usedto make physical contact with the user 105. For example, the drone 170may use the deployed robotic arm 170 b to nudge the user 105 in the leg.The nudge of the user's 105 leg may be performed with a sufficient forcein order to startle the user 105, and urge the user 105 to emerge fromthe immersive virtual reality environment to investigate the cause ofthe nudge. Once the user emerges from the immersive virtual realityenvironment, the user's 105 senses may adjust, and potentially becomeaware of the existence of the potential event (e.g., become aware thathis/her kids are home, become aware of the presence of smoke, or thelike).

The subject matter disclosed by the present specification does notrequire that a drone 170 making contact with the user 105 must includethe drone 170 making physical contact. Instead, the drone 170 may makeother types of contact with the user 105. For example, the drone 170 maymake non-physical contact with the user by using a speaker to outputaudible tones. The outputted audible tones may include the output ofaudible tones that are loud enough for the user 105 to hear the audibletones over the sounds of the immersive virtual reality environment thatis created by the virtual reality device 160. Alternatively, or inaddition, the drone 170 may make non-physical contact with a user 105 bytransmitting a message to the virtual reality device 160 that can bereceived by the virtual reality device 160 and processed to generate, bythe virtual reality device 160, an output that can initiate the user's105 emergence from the virtual reality environment. Alternatively, or inaddition, the drone 170 may make non-physical contact with the user 105by providing a visual output that can be used to initiate the user's 105emergence from a virtual reality environment. For example, the drone 170may navigate to the user's location 105 (or to another location) andgenerate a visual output such as flashing lights that indicates theoccurrence of a potential event. In some implementations, such visualoutput generated by the drone 170 may be intended to alert a person ofthe property who is not immersed in a virtual reality environment sothat the person can communicate (e.g., shout at user 105, physicalcontact user 105, or the like) with the user 105 in a way that will getthe user 105 to emerge from the virtual reality environment. In otherimplementations, the drone 170 can use such visual outputs tocommunicate the occurrence of an event to the user 105 after the useremerges from the virtual reality environment, removes the virtualreality device 160, or a combination thereof.

With reference to the example of FIG. 1, the user 105 is immersed in avirtual reality environment created by the virtual reality device 160.The monitoring system control unit 110 may detect sensor data generatedby sensor 120 h that indicates that the user's kids 107, 109 havearrived home and are going to walk through the front door into Room A.The detected sensor data may include, for example, sensor data generatedand broadcast by a motion sensor indicating that an object has movedthrough the exterior door into Room A. The detected sensor data mayinclude, for example, sensor data generated and broadcast by a contactsensor that indicates an exterior door to Room A has been opened.Alternatively, or in addition, the sensor data may include, for example,sensor data generated and broadcast by a connected doorbell that wasrung by one or more persons standing in front of the exterior door intoRoom A. After detecting the presence of the kids 107, 109, themonitoring system control unit 110 may determine whether the user 105 isimmersed in an immersive virtual reality environment. In response todetecting the family event of the kids 107, 109 coming home anddetermining that the user 105 is immersed in an immersive virtualreality environment, the monitoring system control unit 110 may transmita notification 112 to the virtual reality headset 160 that instructs thevirtual reality headset 160 to display a visual message indicating thatthe user's kids are home. Then, the user 160 can choose to emerge fromthe immersive virtual reality environment, walk to Room A, and greethis/her kids.

In some instances, an initial set of one or more notification alerts maynot be successful in getting the user 105 to emerge from the immersivevirtual reality environment. In such implementations, or otherimplementations that have not had prior failures, the monitoring systemcontrol unit 110 may transmit an instruction to a drone device 170 thatinstructs the drone 170 to begin to navigate along a flight path 172 tothe location of the user 105. The drone device 170 may then travel tothe user 105 along the flight path, and perform an action when the drone170 arrives within a predetermined distance of the user 105. The actionmay include a nudge with a robotic arm, output of loud blaring audiousing a speaker of the drone, output of one or more visual signals(e.g., flashing lights, video message output on a drone display, or thelike), transmitting a message that, when processed by the virtualreality device 160 causes the virtual reality device 160 to output anotification such as an audio output or a video output that initiate theuser's 105 emergence from the virtual reality environment. Though thisexample describes a flying drone, the present disclosure need not be solimited. For example, other non-flying robotic devices may be used suchas a robot capable of navigating an area of the property 101 by walking,rolling, or the like. By way of example, in response to detecting apotential event, the monitoring system control unit 110 may instruct arolling drone to roll along the floor and bump into the user's 105 legin an effort to get the user to emerge from a virtual reality state.

In some implementations, the type of alert notification may be based onthe severity of the event that is associated with alert. For instance,the monitoring system control unit 110 may merely provide the virtualreality headset 160 with an instruction to output an audio alert when alow-risk event is detected. Alternatively, or in addition, for example,the monitoring control unit 110 may initiate deployment of one or moredrones that can be used to navigate to the user's 105 location, andnudge the user 105 in an escalated effort to get the user 105 to emergefrom the immersive virtual reality environment in response to amoderate-risk event or a high-risk event. Alternatively, or in addition,for example, the monitoring system control unit 110 may instruct thevirtual reality headset 160 to power down in response to the detectionof a high-risk event.

The severity of the alert may be determined in a variety of differentways. For example, the system may determine a severity of the eventbased on a calculated severity score. The severity score may be based ona variety of factors including the obtained sensor data (e.g., (i)sensor data indicating biometric data of the user such as a user's heartrate, breathing rate, or the like, (ii) sensor data describing sensedattributes of the property such as number of moving objects, or (iii)the like), the type of detected event (e.g., an alarm events, anemergency events, family events, calendar events, or the like), thestate of the monitoring system (e.g., armed, not armed, number of eventsdetected, number of alarm events detected, number of emergency eventsdetected, or the like), whether one or more previous alerts were sent tothe VR device, the time of day, or the like. For example, a lowerseverity score may be associated with a family event and a higherseverity score may be associated with an emergency event. In suchinstances, the monitoring system control unit may generate instructionsfor a less intrusive alert such as instructions that cause vibration ofthe VR headset for a calendar event (e.g., kids arriving home fromschool). On the other hand, the monitoring system control unit maygenerate instructions for a more intrusive alert such generation of avisual output on the display of the VR headset for an alarm event thatis based on the sensor data sensing motion in a portion of theproperty). By way of yet another example, the monitoring system controlunit may generate instructions for an even more intrusive alert such aspowering down of the VR device in response to an emergency event such asa fire. Alternatively, however, the monitoring system control unit mayonly generate instructions for the rather intrusive alert of poweringdown the VR device when a severe event is detected and the user has notresponded to one or more prior alerts transmitted to the VR device.

The monitoring system control unit 110 can evaluate the severity scoreagainst one or more predetermined thresholds to determine a type ofalert that should be selected. If the severity score does not satisfy afirst severity threshold, then a non-intrusive alert such as a vibrationof the virtual reality headset 160 may be selected. Alternatively, ifthe severity score satisfies a first severity threshold, then a moreintrusive alert may be selected such as an audio message. Additionalseverity thresholds may also be set for different types of alerts suchas video alerts, and power shut down alerts. For example, if theseverity score is satisfies both the first severity threshold and asecond severity threshold, then a video alert may be selected. Finally,if the severity score satisfies a first severity threshold, a secondseverity threshold, and a third severity threshold, then monitoringsystem control unit may select a power shutdown alert. Differentthresholds having a different hierarchical order may also be selected,as this hierarchy of thresholds is just one example of an alert severitythreshold hierarchy.

In some implementations, the monitoring system control unit 110 may alsoevaluate the detected event to determine whether a user should even bealerted to the occurrence of the event. For example, the monitoringsystem control unit 110 be configured to determine the importance of adetected event. If the event is determined to have more than apredetermined level of importance, then the monitoring system controlunit 110 may generate an alert that includes one or more instructions,that when processed by the virtual reality headset 160, alert the user105 to the occurrence of the event. Alternatively, if the event isdetermined to have less than a predetermined level of significance, thenthe monitoring system control unit 110 will not generate an alert thatcan be used to alert the user 105 as to the occurrence of the event.This provides the advantage of reducing the amount of interruptions tothe user's 105 virtual reality experience and only alerting the user 105to the occurrence of events that are sufficiently important.

The monitoring system control unit 110 may determine a level ofimportance associated with the occurrence of a detected event based atleast in part on a calculated importance score. The importance score maybe based on the obtained sensor data (e.g., number of moving objects),the type of detected event (e.g., an alarm events, an emergency events,family events, calendar events, or the like), the duration of thedetected event (e.g., quick movement of an animal darting across theproperty or consistent movement around a window or door of a property),the state of the monitoring system (e.g., armed, not armed, number ofevents detected, number of alarm events detected, number of emergencyevents detected, or the like), the time of day, or the like.

The monitoring system control unit 110 can evaluate the importance scoreagainst a predetermined threshold. If the importance score is determinedto satisfy the predetermined threshold, then the monitoring systemcontrol unit 110 can determine to generate an alert message that can betransmitted to the virtual reality device 160 and processed by thevirtual reality device 160 to output an alert. Alternatively, if theimportance score is determined to not satisfy the predeterminedthreshold, then the monitoring system control unit 110 can determine tonot generate an alert message to the virtual reality device 160 andavoid interrupting the user's 105 virtual reality experience.

The importance score, the severity score, or both, can therefore be usedto intelligently alert a user 105 immersed in a virtual realityenvironment of the detection of the occurrence of an event.

The example of FIG. 1 describes a system where a monitoring systemcontrol unit 110 notifies equipment creating a virtual realityenvironment of the presence of an event. However, the present disclosureneed not be so limited. For instance, the monitoring system may also beconfigured to notify equipment that creates augment realityenvironments, holography environments, or the like. For example, use ofa monitoring system control unit 110 to notify equipment creating anaugmented reality environment of the presence of an event also fallswithin the scope of the present disclosure. For example, use of amonitoring system control unit 110 to notify equipment creating anaugmented reality environment of the presence of an event, andinstructing the augmented reality system to get the user 105 to emergefrom the augmented reality environment also falls within the scope ofthe present disclosure.

FIG. 2 is another contextual diagram of a system 200 for alerting a user205 immersed in a virtual reality environment of the occurrence of anevent that was detected at a property. The system 200 is substantiallythe same as the system 100 except that the system 200 is an example ofthe system notifying a user 205 to get the user 205 to emerge from animmersive virtual reality environment generated by virtual realityheadset 260 in response to the detection of an emergency event such as afire.

With reference to the example of FIG. 2, assume that the user 205 wasusing an iron 280 prior to immersing himself/herself in a virtualreality environment. In the user's 205 rush to enter the virtual realityenvironment, the user 205 forgot to turn off the iron 280, left the ironplugged in 282, and placed the iron 280 face down on the ironing board284. Then, the user 205 uses the virtual reality headset 260 to immersehimself/herself into an immersive virtual reality environment.

The monitoring system control unit 210 may detect sensor data generatedby sensor 220 a that indicates that there is a fire 286 in Room A. Forexample, the sensor 220 a may detect increased temperatures, smoke, or acombination thereof. The monitoring system control unit 210 maydetermine that the user 205 is immersed in a virtual reality environmentbecause obtained sensor data indicates that the user 205 is in anadjacent Room D, is associated with biometric data indicating that theuser 205 is alive and awake, and that the user 205 is showing noreaction to the fire 286. In response to the detection of the fire 286,the monitoring system control unit 210 may transmit a notification 212to the virtual reality headset 260 that instructs the virtual realityheadset 260 to power down immediately based on the high risk threat of afire 286. Then, the user 260 can immediately emerge from the immersivevirtual reality environment, and take necessary steps to use a fireextinguisher to put out the fire 286, exit the property, or the like.

Alternatively, the monitoring system control unit 210 may transmit aninstruction to a drone device 270 that instructs the drone 270 tonavigate a flight path 272 to the location of the user 205. The dronedevice 270 may then travel to the user 205 using the flight path, andperform an action. The action may include a nudge with a robotic arm,output of loud blaring audio using a speaker of the drone, or the like.In some implementations, a drone 270 may be used by the system 200because, for example, the monitoring system control unit 210 is unableto communicate with the virtual reality headset 260, is unsuccessful ingetting the user 205 to emerge from the immersive virtual realityenvironment, or the like. In severe scenarios, the monitoring systemcontrol unit 210 may be configured to deploy multiple drones that caneach be used to make physical contact with the user 205 in an effort toget the user 205 to emerge from the immersive virtual realityenvironment.

Though this example describes a flying drone, the present disclosureneed not be so limited. For example, other non-flying robotic devicesmay be used such as a robot capable of navigating an area of theproperty 201 by walking, rolling, or the like. By way of example, inresponse to detecting a potential event, the monitoring system controlunit 210 may instruct a rolling drone to roll along the floor and bumpinto the user's 205 leg in an effort to wake the user from a virtualreality state.

Though the system 200 is described above with the monitoring systemcontrol unit 210 initiating actions to notify equipment creating avirtual reality environment only after the monitoring system controlunit 210 determines that the user 205 is immersed in a virtual realityenvironment, the present disclosure need not be so limited. In someimplementations, the monitoring system control unit 210 can initiateactions to notify equipment creating a virtual reality environment (orthe drone 270) as soon as a potential event is detected. By way ofexample, upon the detection of more than a threshold amount of carbonmonoxide in the air, the monitoring system control unit 110 mayimmediately transmit a notification to equipment creating a virtualreality environment to instruct the equipment to initiate a user's 205emergence from the virtual reality environment without first determiningwhether the user 205 is immerged in a virtual reality environment.Similarly, upon the detection of a fire, the monitoring system controlunit 210 may immediately transmit a notification to the drone 270 tonudge the user 205 to initiate the user's 205 emergence from the virtualreality environment without first determining whether the user 205 isimmerged in a virtual reality environment.

The example of FIG. 2 describes a system where a monitoring systemcontrol unit 210 notifies equipment creating a virtual realityenvironment of the presence of an event. However, the present disclosureneed not be so limited. For instance, the monitoring system may also beconfigured to notify equipment that creates augment realityenvironments, holography environments, or the like. For example, use ofa monitoring system control unit 210 to notify equipment creating anaugmented reality environment of the presence of an event, andinstructing the augmented reality system to get the user 205 to emergefrom the augmented reality environment also falls within the scope ofthe present disclosure.

FIG. 3 is a flowchart of a process for alerting a user immersed in avirtual reality environment of the occurrence of an event that wasdetected at a property. Generally, the process 300 includes obtainingsensor data from one or more sensors 310, detecting a potential event320, determining whether a user is immersed in a virtual realityenvironment based on the sensor data 330, generating an alertnotification that can be used to initiate the user's emergence from thevirtual reality experience 340, and transmitting the generated alertnotification 350.

In more detail, a computer such as a monitoring system control unit mayobtain sensor data from one or more sensors at stage 310. Obtainingsensor data from one or more sensors may include, for example, obtainingdata generated by one or more motion sensors, one or more smoke sensors,one or more temperature sensors, one or more biometric sensors, one ormore wearable sensors, a combination thereof, or the like.

The monitoring system control unit may detect a potential event at stage320. In some implementations, the monitoring system control unit maydetect the occurrence of the potential event based on obtained sensorsdata. For example, the monitoring system control unit may analyze theobtained sensor data to infer the likelihood that the data is indicativeof an event. Such events may include, for example, alarm events,emergency events, family events, or the like. In other implementations,the monitoring system control unit may detect the occurrence of thepotential event without analyzing obtained sensor data. For example, themonitoring system control unit may detect the occurrence of a potentialevent by accessing one or more calendar event records associated with auser's calendar. Such events may include calendar events such a workmeeting, a personal meeting, or the like that have been added to auser's calendar.

The monitoring system control unit may determine whether a user isimmersed in a virtual reality environment based on the obtained sensordata at stage 330. Determining whether a user is immersed in a virtualreality environment may include, for example, obtaining sensor datagenerated by one or more sensors located throughout the property andanalyzing the obtained sensor data. For example, the sensor data mayinclude data that is indicative of user using a virtual reality headset.For instance, a user may be present in the property, remainingstationary in a particular portion of property, but moving the user'shead and arms. Alternatively, or in addition, the sensor data my mayinclude data from one or more sensors worn by the user. For example, theuser may wear a smartwatch, or other wearable device, that providessensor data regarding the user's health such as a user's heartbeat.

Alternatively, or in addition, the monitoring system control unit mayalso determine whether a user is immersed in a virtual realityenvironment without analyzing the obtained sensor data. For example, themonitoring system control until may receive status notifications fromthe virtual reality headset via the network that indicate when thevirtual reality headset is being used by the user.

In some implementations, a combination of sensor data from multipledifferent sensors may be analyzed to detect that a user is currentlyimmersed in a virtual reality environment at stage 330. For instance,the monitoring system control unit may analyze (i) sensor data fromsensors that is indicative of the user's presence in a particularlocation of a property, degree of movement, both, or the like, (ii)sensor data from one or more wearable sensors worn by the user that mayprovide, e.g., biometric data (e.g., heartbeat) related to the user, and(iii) sensor data from a sensor such as a smoke detector and determinewhether a user is immersed in a virtual reality environment.

The monitoring system control unit may determine whether the event isimportant enough to generate an alert notification that can be used toinitiate the user's emergence from the virtual reality experience atstage 340. Determining that the event is important enough to generate analert notification may be based on the calculation and evaluation of animportance score for the event. In response to determining that theevent is important enough to generate an alert notification, themonitoring system control unit can generate an alert notification thatcan be used to initiate the user's emergence from the virtual realityexperience. Determining whether the event is important enough may bebased on the comparison of the importance score to a predeterminedthreshold.

The alert notifications may be configured to get the user to emerge fromthe immersive virtual reality environment created by the virtual realityheadset to address the potential event detected by the monitoring systemcontrol unit. In some implementations, the monitoring system controlunit may select a particular type of alert notification for generationbased on the severity of the potential event. For example, themonitoring system control unit may select a particular type of eventbased on a severity score that is calculated for the potential event.

The monitoring system control unit may transmit the generated alertnotification 350. In some implementations, the generated alertnotifications may be transmitted to a virtual reality device such as avirtual reality headset. Alternatively, or in addition, the alert may betransmitted to another device such as a drone, a mobile device (e.g., asmartphone, tablet, etc), or the like.

In some implementations, the drone may receive the transmitted messageand perform one or more operations such as navigating to and poking theuser to get the user to emerge from the virtual reality environment. Inthe same, or other, implementations, the generated notification may betransmitted to a mobile device of a different occupant of the propertyor a mobile device of a registered contact designated by the user of thevirtual reality device. The different occupant or the registered contactmay then take one or more actions to get the user of the virtual realitydevice to emerge from the virtual reality environment. In the cases ofthe drone or mobile device, the generated alert message may be includeone or more instructions that are configured to cause each respectivedevice to perform their respective actions in response to the alert(e.g., drone deploying, navigating to, and poking the user or a mobiledevice generating an audible or visual notification/alert).

FIG. 4 is a block diagram of an example of a security monitoring system400 that may utilize virtual reality components.

A security system is augmented with virtual reality components. Avirtual reality headset allows a user to explore sensed attributes of aproperty in virtual environment such that the user may be guided throughan installation process and investigate an entirety of a home from anupstairs bedroom.

FIG. 4 illustrates an example of an electronic system 400 configured toprovide surveillance and reporting. The electronic system 400 includes anetwork 405, a monitoring system control unit 410, one or more userdevices 440, 450, a monitoring application server 460, and a centralalarm station server 470. In some examples, the network 405 facilitatescommunications between the monitoring system control unit 410, the oneor more user devices 440, 450, the monitoring application server 460,and the central alarm station server 470.

The network 405 is configured to enable exchange of electroniccommunications between devices connected to the network 405. Forexample, the network 405 may be configured to enable exchange ofelectronic communications between the monitoring system control unit410, the one or more user devices 440, 450, the monitoring applicationserver 460, and the central alarm station server 470. The network 405may include, for example, one or more of the Internet, Wide AreaNetworks (WANs), Local Area Networks (LANs), analog or digital wired andwireless telephone networks (e.g., a public switched telephone network(PSTN), Integrated Services Digital Network (ISDN), a cellular network,and Digital Subscriber Line (DSL)), radio, television, cable, satellite,or any other delivery or tunneling mechanism for carrying data. Network405 may include multiple networks or subnetworks, each of which mayinclude, for example, a wired or wireless data pathway. The network 405may include a circuit-switched network, a packet-switched data network,or any other network able to carry electronic communications (e.g., dataor voice communications). For example, the network 405 may includenetworks based on the Internet protocol (IP), asynchronous transfer mode(ATM), the PSTN, packet-switched networks based on IP, X.25, or FrameRelay, or other comparable technologies and may support voice using, forexample, VoIP, or other comparable protocols used for voicecommunications. The network 405 may include one or more networks thatinclude wireless data channels and wireless voice channels. The network405 may be a wireless network, a broadband network, or a combination ofnetworks including a wireless network and a broadband network.

The monitoring system control unit 410 includes a controller 412 and anetwork module 414. The controller 412 is configured to control amonitoring system (e.g., a home alarm or security system) that includesthe monitoring system control unit 410. In some examples, the controller412 may include a processor or other control circuitry configured toexecute instructions of a program that controls operation of an alarmsystem. In these examples, the controller 412 may be configured toreceive input from sensors, detectors, or other devices included in thealarm system and control operations of devices included in the alarmsystem or other household devices (e.g., a thermostat, an appliance,lights, etc.). For example, the controller 412 may be configured tocontrol operation of the network module 414 included in the monitoringsystem control unit 410.

The network module 414 is a communication device configured to exchangecommunications over the network 405. The network module 414 may be awireless communication module configured to exchange wirelesscommunications over the network 405. For example, the network module 414may be a wireless communication device configured to exchangecommunications over a wireless data channel and a wireless voicechannel. In this example, the network module 414 may transmit alarm dataover a wireless data channel and establish a two-way voice communicationsession over a wireless voice channel. The wireless communication devicemay include one or more of a LTE module, a GSM module, a radio modem,cellular transmission module, or any type of module configured toexchange communications in one of the following formats: LTE, GSM orGPRS, CDMA, EDGE or EGPRS, EV-DO or EVDO, UMTS, or IP.

The network module 414 also may be a wired communication moduleconfigured to exchange communications over the network 405 using a wiredconnection. For instance, the network module 414 may be a modem, anetwork interface card, or another type of network interface device. Thenetwork module 414 may be an Ethernet network card configured to enablethe monitoring system control unit 410 to communicate over a local areanetwork and/or the Internet. The network module 414 also may be avoiceband modem configured to enable the alarm panel to communicate overthe telephone lines of Plain Old Telephone Systems (POTS).

The monitoring system that includes the monitoring system control unit410 includes one or more sensors or detectors. For example, themonitoring system may include multiple sensors 420. The sensors 420 mayinclude a contact sensor, a motion sensor, a glass break sensor, or anyother type of sensor included in an alarm system or security system. Thesensors 420 also may include an environmental sensor, such as atemperature sensor, a water sensor, a rain sensor, a wind sensor, alight sensor, a smoke detector, a carbon monoxide detector, an airquality sensor, etc. The sensors 420 further may include a healthmonitoring sensor, such as a prescription bottle sensor that monitorstaking of prescriptions, a blood pressure sensor, a blood sugar sensor,a bed mat configured to sense presence of liquid (e.g., bodily fluids)on the bed mat, etc. In some examples, the sensors 420 may include aradio-frequency identification (RFID) sensor that identifies aparticular article that includes a pre-assigned RFID tag.

The monitoring system control unit 410 communicates with the module 422and the camera 430 to perform surveillance or monitoring. The module 422is connected to one or more devices that enable home automation control.For instance, the module 422 may be connected to one or more lightingsystems and may be configured to control operation of the one or morelighting systems. Also, the module 422 may be connected to one or moreelectronic locks at the property and may be configured to controloperation of the one or more electronic locks (e.g., control Z-Wavelocks using wireless communications in the Z-Wave protocol. Further, themodule 422 may be connected to one or more appliances at the propertyand may be configured to control operation of the one or moreappliances. The module 422 may include multiple modules that are eachspecific to the type of device being controlled in an automated manner.The module 422 may control the one or more devices based on commandsreceived from the monitoring system control unit 410. For instance, themodule 422 may cause a lighting system to illuminate an area to providea better image of the area when captured by a camera 430.

The camera 430 may be a video/photographic camera or other type ofoptical sensing device configured to capture images. For instance, thecamera 430 may be configured to capture images of an area within abuilding monitored by the monitoring system control unit 410. The camera430 may be configured to capture single, static images of the area andalso video images of the area in which multiple images of the area arecaptured at a relatively high frequency (e.g., thirty images persecond). The camera 430 may be controlled based on commands receivedfrom the monitoring system control unit 410.

The camera 430 may be triggered by several different types oftechniques. For instance, a Passive Infra Red (PIR) motion sensor may bebuilt into the camera 430 and used to trigger the camera 430 to captureone or more images when motion is detected. The camera 430 also mayinclude a microwave motion sensor built into the camera and used totrigger the camera 430 to capture one or more images when motion isdetected. The camera 430 may have a “normally open” or “normally closed”digital input that can trigger capture of one or more images whenexternal sensors (e.g., the sensors 420, PIR, door/window, etc.) detectmotion or other events. In some implementations, the camera 430 receivesa command to capture an image when external devices detect motion oranother potential alarm event. The camera 430 may receive the commandfrom the controller 412 or directly from one of the sensors 420.

In some examples, the camera 430 triggers integrated or externalilluminators (e.g., Infra Red, Z-wave controlled “white” lights, lightscontrolled by the module 422, etc.) to improve image quality when thescene is dark. An integrated or separate light sensor may be used todetermine if illumination is desired and may result in increased imagequality.

The camera 430 may be programmed with any combination of time/dayschedules, system “arming state”, or other variables to determinewhether images should be captured or not when triggers occur. The camera430 may enter a low-power mode when not capturing images. In this case,the camera 430 may wake periodically to check for inbound messages fromthe controller 412. The camera 430 may be powered by internal,replaceable batteries if located remotely from the monitoring controlunit 410. The camera 430 may employ a small solar cell to recharge thebattery when light is available. Alternatively, the camera 430 may bepowered by the controller's 412 power supply if the camera 430 isco-located with the controller 412.

In some implementations, the camera 430 communicates directly with themonitoring application server 460 over the Internet. In theseimplementations, image data captured by the camera 430 does not passthrough the monitoring system control unit 410 and the camera 430receives commands related to operation from the monitoring applicationserver 460.

The system 400 also includes thermostat 434 to perform dynamicenvironmental control at the property. The thermostat 434 is configuredto monitor temperature and/or energy consumption of an HVAC systemassociated with the thermostat 434, and is further configured to providecontrol of environmental (e.g., temperature) settings. In someimplementations, the thermostat 434 can additionally or alternativelyreceive data relating to activity at a property and/or environmentaldata at a property, e.g., at various locations indoors and outdoors atthe property. The thermostat 434 can directly measure energy consumptionof the HVAC system associated with the thermostat, or can estimateenergy consumption of the HVAC system associated with the thermostat434, for example, based on detected usage of one or more components ofthe HVAC system associated with the thermostat 434. The thermostat 434can communicate temperature and/or energy monitoring information to orfrom the monitoring system control unit 410 and can control theenvironmental (e.g., temperature) settings based on commands receivedfrom the monitoring system control unit 410.

In some implementations, the thermostat 434 is a dynamicallyprogrammable thermostat and can be integrated with the monitoring systemcontrol unit 410. For example, the dynamically programmable thermostat434 can include the monitoring system control unit 410, e.g., as aninternal component to the dynamically programmable thermostat 434. Inaddition, the monitoring system control unit 410 can be a gateway devicethat communicates with the dynamically programmable thermostat 434.

A module 437 is connected to one or more components of an HVAC systemassociated with a property, and is configured to control operation ofthe one or more components of the HVAC system. In some implementations,the module 437 is also configured to monitor energy consumption of theHVAC system components, for example, by directly measuring the energyconsumption of the HVAC system components or by estimating the energyusage of the one or more HVAC system components based on detecting usageof components of the HVAC system. The module 437 can communicate energymonitoring information and the state of the HVAC system components tothe thermostat 434 and can control the one or more components of theHVAC system based on commands received from the thermostat 434.

In some examples, the system 400 further includes one or more roboticdevices. The robotic devices may be any type of robots that are capableof moving and taking actions that assist in security monitoring. Forexample, the robotic devices may include drones that are capable ofmoving throughout a property based on automated control technologyand/or user input control provided by a user. In this example, thedrones may be able to fly, roll, walk, or otherwise move about theproperty. The drones may include helicopter type devices (e.g., quadcopters), rolling helicopter type devices (e.g., roller copter devicesthat can fly and also roll along the ground, walls, or ceiling) and landvehicle type devices (e.g., automated cars that drive around aproperty). In some cases, the robotic devices may be robotic devicesthat are intended for other purposes and merely associated with themonitoring system 400 for use in appropriate circumstances. Forinstance, a robotic vacuum cleaner device may be associated with themonitoring system 400 as one of the robotic devices and may becontrolled to take action responsive to monitoring system events.

In some examples, the robotic devices automatically navigate within aproperty. In these examples, the robotic devices include sensors andcontrol processors that guide movement of the robotic devices within theproperty. For instance, the robotic devices may navigate within theproperty using one or more cameras, one or more proximity sensors, oneor more gyroscopes, one or more accelerometers, one or moremagnetometers, a global positioning system (GPS) unit, an altimeter, oneor more sonar or laser sensors, and/or any other types of sensors thataid in navigation about a space. The robotic devices may include controlprocessors that process output from the various sensors and control therobotic devices to move along a path that reaches the desireddestination and avoids obstacles. In this regard, the control processorsdetect walls or other obstacles in the property and guide movement ofthe robotic devices in a manner that avoids the walls and otherobstacles.

In addition, the robotic devices may store data that describesattributes of the property. For instance, the robotic devices may storea floorplan and/or a three-dimensional model of the property thatenables the robotic devices to navigate the property. During initialconfiguration, the robotic devices may receive the data describingattributes of the property, determine a frame of reference to the data(e.g., a home or reference location in the property), and navigate theproperty based on the frame of reference and the data describingattributes of the property. Further, initial configuration of therobotic devices also may include learning of one or more navigationpatterns in which a user provides input to control the robotic devicesto perform a specific navigation action (e.g., fly to an upstairsbedroom and spin around while capturing video and then return to a homecharging base). In this regard, the robotic devices may learn and storethe navigation patterns such that the robotic devices may automaticallyrepeat the specific navigation actions upon a later request.

In some examples, the robotic devices may include data capture andrecording devices. In these examples, the robotic devices may includeone or more cameras, one or more motion sensors, one or moremicrophones, one or more biometric data collection tools, one or moretemperature sensors, one or more humidity sensors, one or more air flowsensors, and/or any other types of sensors that may be useful incapturing monitoring data related to the property and users in theproperty. The one or more biometric data collection tools may beconfigured to collect biometric samples of a person in the home with orwithout contact of the person. For instance, the biometric datacollection tools may include a fingerprint scanner, a hair samplecollection tool, a skin cell collection tool, and/or any other tool thatallows the robotic devices to take and store a biometric sample that canbe used to identify the person (e.g., a biometric sample with DNA thatcan be used for DNA testing).

In some implementations, the robotic devices may include output devices.In these implementations, the robotic devices may include one or moredisplays, one or more speakers, and/or any type of output devices thatallow the robotic devices to communicate information to a nearby user.

The robotic devices also may include a communication module that enablesthe robotic devices to communicate with the monitoring system controlunit 410, each other, and/or other devices. The communication module maybe a wireless communication module that allows the robotic devices tocommunicate wirelessly. For instance, the communication module may be aWi-Fi module that enables the robotic devices to communicate over alocal wireless network at the property. The communication module furthermay be a 900 MHz wireless communication module that enables the roboticdevices to communicate directly with the monitoring system control unit410. Other types of short-range wireless communication protocols, suchas Bluetooth, Bluetooth LE, Zwave, Zigbee, etc., may be used to allowthe robotic devices to communicate with other devices in the property.

The robotic devices further may include processor and storagecapabilities. The robotic devices may include any suitable processingdevices that enable the robotic devices to operate applications andperform the actions described throughout this disclosure. In addition,the robotic devices may include solid state electronic storage thatenables the robotic devices to store applications, configuration data,collected sensor data, and/or any other type of information available tothe robotic devices.

The robotic devices are associated with one or more charging stations.The charging stations may be located at predefined home base orreference locations in the property. The robotic devices may beconfigured to navigate to the charging stations after completion oftasks needed to be performed for the monitoring system 400. Forinstance, after completion of a monitoring operation or upon instructionby the monitoring system control unit 410, the robotic devices may beconfigured to automatically fly to and land on one of the chargingstations. In this regard, the robotic devices may automatically maintaina fully charged battery in a state in which the robotic devices areready for use by the monitoring system 400.

The charging stations may be contact based charging stations and/orwireless charging stations. For contact based charging stations, therobotic devices may have readily accessible points of contact that therobotic devices are capable of positioning and mating with acorresponding contact on the charging station. For instance, ahelicopter type robotic device may have an electronic contact on aportion of its landing gear that rests on and mates with an electronicpad of a charging station when the helicopter type robotic device landson the charging station. The electronic contact on the robotic devicemay include a cover that opens to expose the electronic contact when therobotic device is charging and closes to cover and insulate theelectronic contact when the robotic device is in operation.

For wireless charging stations, the robotic devices may charge through awireless exchange of power. In these cases, the robotic devices needonly locate themselves closely enough to the wireless charging stationsfor the wireless exchange of power to occur. In this regard, thepositioning needed to land at a predefined home base or referencelocation in the property may be less precise than with a contact basedcharging station. Based on the robotic devices landing at a wirelesscharging station, the wireless charging station outputs a wirelesssignal that the robotic devices receive and convert to a power signalthat charges a battery maintained on the robotic devices.

In some implementations, each of the robotic devices has a correspondingand assigned charging station such that the number of robotic devicesequals the number of charging stations. In these implementations, therobotic devices always navigate to the specific charging stationassigned to that robotic device. For instance, a first robotic devicemay always use a first charging station and a second robotic device mayalways use a second charging station.

In some examples, the robotic devices may share charging stations. Forinstance, the robotic devices may use one or more community chargingstations that are capable of charging multiple robotic devices. Thecommunity charging station may be configured to charge multiple roboticdevices in parallel. The community charging station may be configured tocharge multiple robotic devices in serial such that the multiple roboticdevices take turns charging and, when fully charged, return to apredefined home base or reference location in the property that is notassociated with a charger. The number of community charging stations maybe less than the number of robotic devices.

Also, the charging stations may not be assigned to specific roboticdevices and may be capable of charging any of the robotic devices. Inthis regard, the robotic devices may use any suitable, unoccupiedcharging station when not in use. For instance, when one of the roboticdevices has completed an operation or is in need of battery charge, themonitoring system control unit 410 references a stored table of theoccupancy status of each charging station and instructs the roboticdevice to navigate to the nearest charging station that is unoccupied.

The system 400 further includes one or more virtual reality devices 480.The one or more virtual reality devices 480 may include any type ofdevice allowing a user to immerse themselves in an environment thatsimulates a physical presence in one or more places. For instance, theone or more virtual reality devices 480 may include an input/outputinterface that allows a user to interact with the environment. The oneor more virtual reality devices 480 may include output devices forproviding sensory experience to the user, such as displays and speakers,and input devices for controlling one or more aspects of the experiencebased on user input, such as sensors and mechanical controls (e.g.,buttons). For example, the one or more virtual reality devices 480 mayinclude one or more wearable virtual reality head-mounted displays orheadsets that may be worn by a user.

In some implementations, the one or more virtual reality devices 480 maysimulate a physical presence in one or more places located within oraround a property monitored by system 400. The one or more virtualreality devices 480 may provide a user with this experience bycommunicating with one or more components of system 400, such as sensors420, module 422, and camera 430. For instance, a user may be able toview a live feed from camera 430 on a display of a virtual realityheadset 480 worn by the user. In some implementations, the virtualreality headset 480 may monitor data from one or more accelerometersincluded in the virtual reality headset 480 and control panning,tilting, and/or zooming functions of camera 430 based on the user's headmovements. This may allow for synchronous movement of camera 430 and thevirtual reality headset 480 as positioned on the user's head, which mayprovide the user with the sensation of physical presence within the areamonitored by camera 430. The virtual reality headset 480 may provide oneor more control signals to camera 430 such that it tilts and pans withmovement of the user's head. Zoom functions of camera 430 may becontrolled in accordance with leaning movements of the user's head. Insome implementations, the virtual reality headset 480 may determine apositioning of the user's head in three-dimensional space and controlmovement of camera 430 along each axis in accordance withDenavit-Hartenberg parameters.

In some implementations, the virtual reality headset 480 may controlzoom functions of camera 430 based on the user's eye movements asdetected by a camera integral with the virtual reality headset 480. Inthese implementations, the virtual reality headset 480 may determinethat the user's eyes are focused on a particular portion of its displayand/or object depicted in an image of its display and, in response,provide camera 430 with the control signals necessary to zoom into theuser's area of interest. In these implementations, the virtual realitydevice 480 may “lock-onto” an object included in images provided bycamera 430 in response to determining that the user has shown arelatively high degree of interest in the object. For instance, thevirtual reality device 480 may identify one or more objects included inthe images of camera 430 and track the user's gaze to determine if theuser's eyes have consistently followed one of the identified objects.The virtual reality headset 480 may control camera 430 such that itcontinuously tracks these objects of interest.

In some examples, the one or more virtual reality devices 480 maysimulate a physical navigation of one or more places located within oraround the property monitored by system 400. For instance, if theproperty includes multiple cameras 430, a virtual reality headset 480may seamlessly transition from feed-to-feed to simulate a traversal ofan area monitored by a series of cameras 430. The user may be able topan, tilt, and zoom each camera 430, as described above, as well as“walk” through the environment using one or more directional controlsprovided by head movement or tactile input, for example. That is, thevirtual reality headset 480 may coordinate movements of cameras 430 andthe feed provided for display for the user, based on the user'smovements, directional input, and the location of each camera 430, tosimulate movement and translate the user's vantage point. For example,this may allow a user to investigate a potential threat on theirproperty from a remote location. When utilized in combination with the“lock-on” feature described above, a user may be able to follow a movingobject located within or around the property.

In some implementations, the system 400 includes one or morethree-dimensional scanners or one or more depth sensors. In theseimplementations, the one or more virtual reality devices 480 may be ableto provide the user with a three-dimensional simulation of one or moreareas of their property. When utilized in combination with thetechniques described above in association with cameras 430, the user maybe able to interact with three-dimensional renderings of objects on theproperty. In some implementations, data from the one or morethree-dimensional scanners or one or more depth sensors may be utilizedto model an environment, while data from one or more cameras 430 may beutilized to texturize the models generated. This may provide the userwith a very realistic experience, allowing them to quickly andaccurately identify and assess threats. In some implementations, thethree-dimensional or depth information is gleaned from data provided byone or more sensors 420 and/or cameras 430.

The one or more virtual reality devices 480 may also provide virtualinstructions to the user. For instance, the one or more virtual realitydevices 480 may help guide the user through one or more installationprocesses associated with system 400. The one or more virtual realitydevices 480 may communicate with a remotely-located technician, who willbe able to receive a view of the user's home and provide instructionaccordingly. For example, a technician may be able to see what the useris seeing and augment the display of their virtual reality headset 480with one or more images to provide a demonstration for the user andassist with system configuration, installation, and/or maintenance.

The one or more virtual reality devices 480 may include one or morecameras, one or more proximity sensors, one or more gyroscopes, one ormore accelerometers, one or more magnetometers, a global positioningsystem (GPS) unit, an altimeter, one or more sonar or laser sensors, oneor more motion sensors, one or more accelerometers, one or more buttonsor other mechanical controls, one or more microphones, and/or any othertypes of sensors. Furthermore, the one or more virtual reality devices480 may receive input from any of sensors 420.

The sensors 420, the module 422, the camera 430, the thermostat 434, andthe virtual reality devices 480 communicate with the controller 412 overcommunication links 424, 426, 428, 432, 484, and 486. The communicationlinks 424, 426, 428, 432, 484, and 486 may be a wired or wireless datapathway configured to transmit signals from the sensors 420, the module422, the camera 430, the thermostat 434, and the virtual reality devices480 to the controller 412. The sensors 420, the module 422, the camera430, the thermostat 434, and the virtual reality devices 480 maycontinuously transmit sensed values to the controller 412, periodicallytransmit sensed values to the controller 412, or transmit sensed valuesto the controller 412 in response to a change in a sensed value.

The communication links 424, 426, 428, 432, 484, and 486 may include alocal network. The sensors 420, the module 422, the camera 430, thethermostat 434, and the virtual reality devices 480 and the controller412 may exchange data and commands over the local network. The localnetwork may include 802.11 “WiFi” wireless Ethernet (e.g., usinglow-power WiFi chipsets), Z-Wave, Zigbee, Bluetooth, “Homeplug” or other“Powerline” networks that operate over AC wiring, and a Category 5(CATS) or Category 6 (CAT6) wired Ethernet network. The local networkmay be a mesh network constructed based on the devices connected to themesh network.

The monitoring application server 460 is an electronic device configuredto provide monitoring services by exchanging electronic communicationswith the monitoring system control unit 410, the one or more userdevices 440, 450, and the central alarm station server 470 over thenetwork 405. For example, the monitoring application server 460 may beconfigured to monitor events (e.g., alarm events) generated by themonitoring system control unit 410. In this example, the monitoringapplication server 460 may exchange electronic communications with thenetwork module 414 included in the monitoring system control unit 410 toreceive information regarding events (e.g., alarm events) detected bythe monitoring system control unit 110. The monitoring applicationserver 460 also may receive information regarding events (e.g., alarmevents) from the one or more user devices 440, 450.

In some examples, the monitoring application server 460 may route alarmdata received from the network module 414 or the one or more userdevices 440, 450 to the central alarm station server 470. For example,the monitoring application server 460 may transmit the alarm data to thecentral alarm station server 470 over the network 405.

The monitoring application server 460 may store sensor and image datareceived from the monitoring system and perform analysis of sensor andimage data received from the monitoring system. Based on the analysis,the monitoring application server 460 may communicate with and controlaspects of the monitoring system control unit 410 or the one or moreuser devices 440, 450.

The central alarm station server 470 is an electronic device configuredto provide alarm monitoring service by exchanging communications withthe monitoring system control unit 410, the one or more mobile devices440, 450, and the monitoring application server 460 over the network405. For example, the central alarm station server 470 may be configuredto monitor alarm events generated by the monitoring system control unit410. In this example, the central alarm station server 470 may exchangecommunications with the network module 414 included in the monitoringsystem control unit 410 to receive information regarding alarm eventsdetected by the monitoring system control unit 410. The central alarmstation server 470 also may receive information regarding alarm eventsfrom the one or more mobile devices 440, 450 and/or the monitoringapplication server 460.

The central alarm station server 470 is connected to multiple terminals472 and 474. The terminals 472 and 474 may be used by operators toprocess alarm events. For example, the central alarm station server 470may route alarm data to the terminals 472 and 474 to enable an operatorto process the alarm data. The terminals 472 and 474 may includegeneral-purpose computers (e.g., desktop personal computers,workstations, or laptop computers) that are configured to receive alarmdata from a server in the central alarm station server 470 and render adisplay of information based on the alarm data. For instance, thecontroller 412 may control the network module 414 to transmit, to thecentral alarm station server 470, alarm data indicating that a sensor420 detected a door opening when the monitoring system was armed. Thecentral alarm station server 470 may receive the alarm data and routethe alarm data to the terminal 472 for processing by an operatorassociated with the terminal 472. The terminal 472 may render a displayto the operator that includes information associated with the alarmevent (e.g., the name of the user of the alarm system, the address ofthe building the alarm system is monitoring, the type of alarm event,etc.) and the operator may handle the alarm event based on the displayedinformation.

In some implementations, the terminals 472 and 474 may be mobile devicesor devices designed for a specific function. Although FIG. 4 illustratestwo terminals for brevity, actual implementations may include more (and,perhaps, many more) terminals.

The one or more user devices 440, 450 are devices that host and displayuser interfaces. For instance, the user device 440 is a mobile devicethat hosts one or more native applications (e.g., the nativesurveillance application 442). The user device 440 may be a cellularphone or a non-cellular locally networked device with a display. Theuser device 440 may include a cell phone, a smart phone, a tablet PC, apersonal digital assistant (“PDA”), or any other portable deviceconfigured to communicate over a network and display information. Forexample, implementations may also include Blackberry-type devices (e.g.,as provided by Research in Motion), electronic organizers, iPhone-typedevices (e.g., as provided by Apple), iPod devices (e.g., as provided byApple) or other portable music players, other communication devices, andhandheld or portable electronic devices for gaming, communications,and/or data organization. The user device 440 may perform functionsunrelated to the monitoring system, such as placing personal telephonecalls, playing music, playing video, displaying pictures, browsing theInternet, maintaining an electronic calendar, etc.

The user device 440 includes a native surveillance application 442. Thenative surveillance application 442 refers to a software/firmwareprogram running on the corresponding mobile device that enables the userinterface and features described throughout. The user device 440 mayload or install the native surveillance application 442 based on datareceived over a network or data received from local media. The nativesurveillance application 442 runs on mobile devices platforms, such asiPhone, iPod touch, Blackberry, Google Android, Windows Mobile, etc. Thenative surveillance application 442 enables the user device 440 toreceive and process image and sensor data from the monitoring system.

The user device 450 may be a general-purpose computer (e.g., a desktoppersonal computer, a workstation, or a laptop computer) that isconfigured to communicate with the monitoring application server 460and/or the monitoring system control unit 410 over the network 405. Theuser device 450 may be configured to display a surveillance monitoringuser interface 452 that is generated by the user device 450 or generatedby the monitoring application server 460. For example, the user device450 may be configured to display a user interface (e.g., a web page)provided by the monitoring application server 460 that enables a user toperceive images captured by the camera 430 and/or reports related to themonitoring system. Although FIG. 4 illustrates two user devices forbrevity, actual implementations may include more (and, perhaps, manymore) or fewer user devices.

In some implementations, the one or more user devices 440, 450communicate with and receive monitoring system data from the monitoringsystem control unit 410 using the communication link 438. For instance,the one or more user devices 440, 450 may communicate with themonitoring system control unit 410 using various local wirelessprotocols such as WiFi, Bluetooth, Zwave, Zigbee, HomePlug (Ethernetover powerline), or wired protocols such as Ethernet and USB, to connectthe one or more user devices 440, 450 to local security and automationequipment. The one or more user devices 440, 450 may connect locally tothe monitoring system and its sensors and other devices. The localconnection may improve the speed of status and control communicationsbecause communicating through the network 405 with a remote server(e.g., the monitoring application server 460) may be significantlyslower.

Although the one or more user devices 440, 450 are shown ascommunicating with the monitoring system control unit 410, the one ormore user devices 440, 450 may communicate directly with the sensors andother devices controlled by the monitoring system control unit 410. Insome implementations, the one or more user devices 440, 450 replace themonitoring system control unit 410 and perform the functions of themonitoring system control unit 410 for local monitoring and longrange/offsite communication.

In other implementations, the one or more user devices 440, 450 receivemonitoring system data captured by the monitoring system control unit410 through the network 405. The one or more user devices 440, 450 mayreceive the data from the monitoring system control unit 410 through thenetwork 405 or the monitoring application server 460 may relay datareceived from the monitoring system control unit 410 to the one or moreuser devices 440, 450 through the network 405. In this regard, themonitoring application server 460 may facilitate communication betweenthe one or more user devices 440, 450 and the monitoring system.

In some implementations, the one or more user devices 440, 450 may beconfigured to switch whether the one or more user devices 440, 450communicate with the monitoring system control unit 410 directly (e.g.,through link 438) or through the monitoring application server 460(e.g., through network 405) based on a location of the one or more userdevices 440, 450. For instance, when the one or more user devices 440,450 are located close to the monitoring system control unit 410 and inrange to communicate directly with the monitoring system control unit410, the one or more user devices 440, 450 use direct communication.When the one or more user devices 440, 450 are located far from themonitoring system control unit 410 and not in range to communicatedirectly with the monitoring system control unit 410, the one or moreuser devices 440, 450 use communication through the monitoringapplication server 460.

Although the one or more user devices 440, 450 are shown as beingconnected to the network 405, in some implementations, the one or moreuser devices 440, 450 are not connected to the network 405. In theseimplementations, the one or more user devices 440, 450 communicatedirectly with one or more of the monitoring system components and nonetwork (e.g., Internet) connection or reliance on remote servers isneeded.

In some implementations, the one or more user devices 440, 450 are usedin conjunction with only local sensors and/or local devices in a house.In these implementations, the system 400 only includes the one or moreuser devices 440, 450, the sensors 420, the module 422, the camera 430,and the robotic devices. The one or more user devices 440, 450 receivedata directly from the sensors 420, the module 422, the camera 430, andthe robotic devices and sends data directly to the sensors 420, themodule 422, the camera 430, and the robotic devices. The one or moreuser devices 440, 450 provide the appropriate interfaces/processing toprovide visual surveillance and reporting.

In other implementations, the system 400 further includes network 405and the sensors 420, the module 422, the camera 430, the thermostat 434,and the robotic devices are configured to communicate sensor and imagedata to the one or more user devices 440, 450 over network 405 (e.g.,the Internet, cellular network, etc.). In yet another implementation,the sensors 420, the module 422, the camera 430, the thermostat 434, andthe robotic devices (or a component, such as a bridge/router) areintelligent enough to change the communication pathway from a directlocal pathway when the one or more user devices 440, 450 are in closephysical proximity to the sensors 420, the module 422, the camera 430,the thermostat 434, and the robotic devices to a pathway over network405 when the one or more user devices 440, 450 are farther from thesensors 420, the module 422, the camera 430, the thermostat 434, and therobotic devices. In some examples, the system leverages GPS informationfrom the one or more user devices 440, 450 to determine whether the oneor more user devices 440, 450 are close enough to the sensors 420, themodule 422, the camera 430, the thermostat 434, and the robotic devicesto use the direct local pathway or whether the one or more user devices440, 450 are far enough from the sensors 420, the module 422, the camera430, the thermostat 434, and the robotic devices that the pathway overnetwork 405 is required. In other examples, the system leverages statuscommunications (e.g., pinging) between the one or more user devices 440,450 and the sensors 420, the module 422, the camera 430, the thermostat434, and the robotic devices to determine whether communication usingthe direct local pathway is possible. If communication using the directlocal pathway is possible, the one or more user devices 440, 450communicate with the sensors 420, the module 422, the camera 430, thethermostat 434, and the robotic devices using the direct local pathway.If communication using the direct local pathway is not possible, the oneor more user devices 440, 450 communicate with the sensors 420, themodule 422, the camera 430, the thermostat 434, and the robotic devicesusing the pathway over network 405.

In some implementations, the system 400 provides end users with accessto images captured by the camera 430 to aid in decision making. Thesystem 400 may transmit the images captured by the camera 430 over awireless WAN network to the user devices 440, 450. Because transmissionover a wireless WAN network may be relatively expensive, the system 400uses several techniques to reduce costs while providing access tosignificant levels of useful visual information.

In some implementations, a state of the monitoring system and otherevents sensed by the monitoring system may be used to enable/disablevideo/image recording devices (e.g., the camera 430). In theseimplementations, the camera 430 may be set to capture images on aperiodic basis when the alarm system is armed in an “Away” state, butset not to capture images when the alarm system is armed in a “Stay”state or disarmed. In addition, the camera 430 may be triggered to begincapturing images when the alarm system detects an event, such as analarm event, a door opening event for a door that leads to an areawithin a field of view of the camera 430, or motion in the area withinthe field of view of the camera 430. In other implementations, thecamera 430 may capture images continuously, but the captured images maybe stored or transmitted over a network when needed.

The described systems, methods, and techniques may be implemented indigital electronic circuitry, computer hardware, firmware, software, orin combinations of these elements. Apparatus implementing thesetechniques may include appropriate input and output devices, a computerprocessor, and a computer program product tangibly embodied in amachine-readable storage device for execution by a programmableprocessor. A process implementing these techniques may be performed by aprogrammable processor executing a program of instructions to performdesired functions by operating on input data and generating appropriateoutput. The techniques may be implemented in one or more computerprograms that are executable on a programmable system including at leastone programmable processor coupled to receive data and instructionsfrom, and to transmit data and instructions to, a data storage system,at least one input device, and at least one output device. Each computerprogram may be implemented in a high-level procedural or object-orientedprogramming language, or in assembly or machine language if desired; andin any case, the language may be a compiled or interpreted language.Suitable processors include, by way of example, both general and specialpurpose microprocessors. Generally, a processor will receiveinstructions and data from a read-only memory and/or a random accessmemory. Storage devices suitable for tangibly embodying computer programinstructions and data include all forms of non-volatile memory,including by way of example semiconductor memory devices, such asErasable Programmable Read-Only Memory (EPROM), Electrically ErasableProgrammable Read-Only Memory (EEPROM), and flash memory devices;magnetic disks such as internal hard disks and removable disks;magneto-optical disks; and Compact Disc Read-Only Memory (CD-ROM). Anyof the foregoing may be supplemented by, or incorporated in,specially-designed ASICs (application-specific integrated circuits).

It will be understood that various modifications may be made. Forexample, other useful implementations could be achieved if steps of thedisclosed techniques were performed in a different order and/or ifcomponents in the disclosed systems were combined in a different mannerand/or replaced or supplemented by other components. Accordingly, otherimplementations are within the scope of the disclosure.

The invention claimed is:
 1. A monitoring system, comprising: one ormore processors; and one or more storage devices that includeinstructions that are operable, when executed by the one or moreprocessors, to cause the one or more processors to perform operationscomprising: detecting an occurrence of an event at a property; based ondetecting the event: determining, based on sensor data generated by oneor more sensors, that an occupant of the property is immersed in avirtual reality environment, wherein determining that the occupant ofthe property is immersed in a virtual reality environment comprises:analyzing the sensor data to determine one or more characteristics ofthe occupant; and based on (i) the determined one or morecharacteristics and (ii) the detected event at the property, determiningthat the occupant of the property is immersed in a virtual realityenvironment; and, based on determining that the occupant of the propertyis immersed in the virtual reality environment, generating an alertnotification that, when processed by another device, causes the deviceto perform one or more operations that initiate the occupant's emergencefrom the virtual reality environment.
 2. The monitoring system of claim1, the operations further comprising: transmitting the generated alertnotification to the other device.
 3. The monitoring system of claim 1,the operations further comprising: detecting an occurrence of adifferent event at the property; based on detection of the differentevent: obtaining second sensor data; determining, based on the obtainedsecond sensor data, that the occupant of the property is immersed in thevirtual reality environment, wherein determining that the occupant ofthe property is immersed in the virtual reality environment comprises:analyzing the second sensor data to determine one or morecharacteristics of the occupant; and based on (i) the determined one ormore characteristics determined based on the second sensor data and (ii)the detected different event at the property, determining that theoccupant of the property is immersed in the virtual reality environment;determining an importance score for the detected different event;determining that the importance score for the detected different eventsatisfies a predetermined threshold; and based on determining that theimportance score for the detected different event satisfies thepredetermined threshold, generating the alert notification fortransmission to the other device.
 4. The monitoring system of claim 3,the operations further comprising: detecting the occurrence of thedifferent event at the property; based on detection of the differentevent: obtaining the second sensor data; determining, based on theobtained second sensor data, that the occupant of the property isimmersed in the virtual reality environment, wherein determining thatthe occupant of the property is immersed in the virtual realityenvironment comprises: analyzing the second sensor data to determine oneor more characteristics of the occupant; and based on (i) the determinedone or more characteristics determined based on the second sensor dataand (ii) the detected different event at the property, determining thatthe occupant of the property is immersed in a virtual realityenvironment; determining the importance score for the detected differentevent; determining that the importance score for the detected differentevent does not satisfy the predetermined threshold; and based ondetermining that the importance score for the detected different eventdoes not satisfy the predetermined threshold, determining not togenerate the alert notification for transmission to the other device. 5.The monitoring system of claim 1, wherein the other device includes avirtual reality headset.
 6. The monitoring system of claim 1, whereinthe other device includes a wearable device that is worn by the occupantof the property.
 7. The monitoring system of claim 1, wherein the one ormore sensors include one or more sensors of the monitoring system, theone or more sensors of the monitoring system comprising at least one ofa motion sensor, a camera, or a thermometer.
 8. The monitoring system ofclaim 1, wherein the one or more sensors include one or more sensors ofone or more wearable devices worn by the occupant of the property, theone or more sensors of the one or more wearable devices including aheart rate sensor, a breathing rate sensor, or a temperature sensor. 9.A method comprising: detecting an occurrence of an event at a property;based on detecting the event: determining, based on sensor datagenerated by one or more sensors, that an occupant of the property isimmersed in a virtual reality environment, wherein determining that theoccupant of the property is immersed in a virtual reality environmentcomprises: analyzing the sensor data to determine one or morecharacteristics of the occupant; and based on (i) the determined one ormore characteristics and (ii) the detected event at the property,determining that the occupant of the property is immersed in a virtualreality environment; and, based on determining that the occupant of theproperty is immersed in the virtual reality environment, generating analert notification that, when processed by another device, causes thedevice to perform one or more operations that initiate the occupant'semergence from the virtual reality environment.
 10. The method of claim9, the method further comprising: transmitting the generated alertnotification to the other device.
 11. The method of claim 9, the methodfurther comprising: detecting an occurrence of a different event at theproperty; based on detection of the different event: obtaining secondsensor data; determining, based on the obtained second sensor data, thatthe occupant of the property is immersed in the virtual realityenvironment, wherein determining that the occupant of the property isimmersed in the virtual reality environment comprises: analyzing thesecond sensor data to determine one or more characteristics of theoccupant; and based on (i) the determined one or more characteristicsdetermined based on the second sensor data and (ii) the detecteddifferent event at the property, determining that the occupant of theproperty is immersed in the virtual reality environment; determining animportance score for the detected different event; determining that theimportance score for the detected different event satisfies apredetermined threshold; and based on determining that the importancescore for the detected different event satisfies the predeterminedthreshold, generating the alert notification for transmission to theother device.
 12. The method of claim 9, the method further comprising:detecting an occurrence of a different event at the property; based ondetection of the different event: obtaining second sensor data;determining, based on the obtained second sensor data, that the occupantof the property is immersed in the virtual reality environment, whereindetermining that the occupant of the property is immersed in the virtualreality environment comprises: analyzing the second sensor data todetermine one or more characteristics of the occupant; and based on (i)the determined one or more characteristics determined based on thesecond sensor data and (ii) the detected different event at theproperty, determining that the occupant of the property is immersed inthe virtual reality environment; determining an importance score for thedetected different event; determining that the importance score for thedetected different event does not satisfy a predetermined threshold; andbased on determining that the importance score for the detecteddifferent event does not satisfy the predetermined threshold,determining not to generate the alert notification for transmission tothe other device.
 13. The method of claim 9, wherein the one or moresensors include one or more sensors of a monitoring system, the one ormore sensors of the monitoring system comprising at least one of amotion sensor, a camera, or a thermometer.
 14. The method of claim 9,wherein the one or more sensors include one or more sensors of one ormore wearable devices worn by the occupant of the property, the one ormore sensors of the one or more wearable devices including a heart ratesensor, a breathing rate sensor, or a temperature sensor.
 15. Anon-transitory computer-readable storage device having stored thereoninstructions, which, when executed by a data processing apparatus, causethe data processing apparatus to perform operations comprising:detecting an occurrence of an event at a property; based on detectingthe event: determining, based on sensor data generated by one or moresensors, that an occupant of the property is immersed in a virtualreality environment, wherein determining that the occupant of theproperty is immersed in a virtual reality environment comprises:analyzing the sensor data to determine one or more characteristics ofthe occupant; and based on (i) the determined one or morecharacteristics and (ii) the detected event at the property, determiningthat the occupant of the property is immersed in a virtual realityenvironment; and, based on determining that the occupant of the propertyis immersed in the virtual reality environment, generating an alertnotification that, when processed by another device, causes the deviceto perform one or more operations that initiate the occupant's emergencefrom the virtual reality environment.
 16. The computer-readable storagedevice of claim 15, the operations further comprising: transmitting thegenerated alert notification to the other device.
 17. Thecomputer-readable storage device of claim 15, the operations furthercomprising: detecting an occurrence of a different event at theproperty; based on detection of the different event: obtaining secondsensor data; determining, based on the obtained second sensor data, thatthe occupant of the property is immersed in the virtual realityenvironment, wherein determining that the occupant of the property isimmersed in the virtual reality environment comprises: analyzing thesecond sensor data to determine one or more characteristics of theoccupant; and based on (i) the determined one or more characteristicsdetermined based on the second sensor data and (ii) the detecteddifferent event at the property, determining that the occupant of theproperty is immersed in the virtual reality environment; determining animportance score for the detected different event; determining that theimportance score for the detected different event satisfies apredetermined threshold; and based on determining that the importancescore for the detected different event satisfies the predeterminedthreshold, generating the alert notification for transmission to theother device.
 18. The computer-readable storage device of claim 15, theoperations further comprising: detecting an occurrence of a differentevent at the property; based on detection of the different event:obtaining second sensor data; determining, based on the obtained secondsensor data, that the occupant of the property is immersed in thevirtual reality environment, wherein determining that the occupant ofthe property is immersed in the virtual reality environment comprises:analyzing the second sensor data to determine one or morecharacteristics of the occupant; and based on (i) the determined one ormore characteristics determined based on the second sensor data and (ii)the detected different event at the property, determining that theoccupant of the property is immersed in the virtual reality environment;determining an importance score for the detected different event;determining that the importance score for the detected different eventdoes not satisfy a predetermined threshold; and based on determiningthat the importance score for the detected different event does notsatisfy the predetermined threshold, determining not to generate thealert notification for transmission to the other device.
 19. Thecomputer-readable storage device of claim 15, wherein the one or moresensors include one or more sensors of a monitoring system, the one ormore sensors of the monitoring system comprising at least one of amotion sensor, a camera, or a thermometer.
 20. The computer-readablestorage device of claim 15, wherein the one or more sensors include oneor more sensors of one or more wearable devices worn by the occupant ofthe property, the one or more sensors of the one or more wearabledevices including a heart rate sensor, a breathing rate sensor, or atemperature sensor.